Add Cosmos DB dedicated gateway bypass cache and shard key support by Copilot · Pull Request #44846 · Azure/azure-sdk-for-python

Copilot · 2026-01-26T17:22:47Z

Implements two dedicated gateway features missing from Python SDK, bringing parity with Java SDK: cache bypass control and shard-based routing.

Changes

HTTP Headers

x-ms-dedicatedgateway-bypass-cache - control cache usage per request
x-ms-dedicatedgateway-shard-key - route to specific gateway instances

API Surface
Added optional parameters to read_item(), read_all_items(), and query_items() (sync + async):

bypass_integrated_cache: Optional[bool] - when True, bypasses cache and reads from backend
dedicated_gateway_shard_key: Optional[str] - routes request to instance with cached shard data

Validation

Shard keys restricted to alphanumeric characters and hyphens
Consistent null checking with is not None throughout

Usage

# Bypass cache for this read
container.read_item(
    item="id", 
    partition_key="pk",
    bypass_integrated_cache=True
)

# Route to specific shard instance
container.query_items(
    query="SELECT * FROM c",
    partition_key="pk",
    dedicated_gateway_shard_key="shard-123"
)

References

Java bypass cache: Cache bypass in dedicated gateway request options azure-sdk-for-java#35865
Java shard key: Dedicatedgateway shardkey request option azure-sdk-for-java#47796

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

scanning-api.github.com
- Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

This section details on the original issue you should resolve

<issue_title>Cosmos: additional support for dedicated gateway SQLx features</issue_title>
<issue_description>We are currently missing two dedicated gateway features in the Python SDK:

Bypass Dedicated Gateway
Provide more control to customers by allowing them to choose whether to use the integrated cache or not. If set to true, the read will be served by the backend and won't be cached in the dedicated gateway.

ref: Azure/azure-sdk-for-java#35865

Shard Key
Allow customers to optionally specify a shard key to use the new sharding feature.
Specifying the shard key will help route the request to an instance that has cached data for this shard or bypass the SQLx cache if correlated instance is down. If the shard key is not specified, it will fall back to the default behavior, which is randomly selecting an instance to serve the request.

ref: Azure/azure-sdk-for-java#47796</issue_description>

<agent_instructions>The PRs referenced across this issue should have most if not all of the information you need to put this together. A lot of things have changed within the Python SDK since the first PR on the integrated cache that was linked, so make sure you take that into account.</agent_instructions>

Comments on the Issue (you are @copilot in this section)
@simorenoh Reference PR for the first addition of any integrated cache functionality to the Python SDK: https://github.com//pull/22946

Fixes Cosmos: additional support for dedicated gateway SQLx features #44845

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

- Added HTTP header constants for bypass cache and shard key - Added validation function for shard key (alphanumeric + hyphen only) - Updated GetHeaders to send bypass and shard key headers - Added parameters to sync container methods: read_item, read_all_items, query_items - Added parameters to async container methods with same functionality - Updated docstrings for new parameters Co-authored-by: simorenoh <30335873+simorenoh@users.noreply.github.com>

…eway_shard_key - Fixed bypass cache header to be sent when value is False - Added unit tests for bypass_integrated_cache with True/False/None values - Added unit tests for dedicated_gateway_shard_key validation - Added test for invalid shard key characters - Added test for combined bypass cache and shard key usage in queries - All validation and header generation tests passing Co-authored-by: simorenoh <30335873+simorenoh@users.noreply.github.com>

- Updated all query_items overloads in async container to include new parameters - Ensured consistency across all method signatures - All sync and async methods now support bypass_integrated_cache and dedicated_gateway_shard_key Co-authored-by: simorenoh <30335873+simorenoh@users.noreply.github.com>

- Improved boolean-to-string conversion efficiency in GetHeaders - Updated tests to use pytest.raises() instead of try-except blocks - Applied modern Python testing practices for cleaner exception testing Co-authored-by: simorenoh <30335873+simorenoh@users.noreply.github.com>

- Fixed inconsistent null checking for max_integrated_cache_staleness_in_ms - Changed truthiness check to explicit 'is not None' to handle 0 values correctly - Added missing docstrings for bypass_integrated_cache and dedicated_gateway_shard_key in async read_item - Ensures consistency between sync and async documentation Co-authored-by: simorenoh <30335873+simorenoh@users.noreply.github.com>

- Fixed string quote consistency (use double quotes throughout) - Fixed parameter indentation alignment in tests - All code now follows consistent formatting standards Co-authored-by: simorenoh <30335873+simorenoh@users.noreply.github.com>

Copilot

Pull request overview

Adds missing Cosmos DB dedicated gateway request controls to the Python SDK to match Java capabilities, enabling per-request integrated cache bypass and shard-based routing via new HTTP headers.

Changes:

Introduces dedicated gateway headers for cache bypass and shard key routing and wires them through request option handling.
Extends sync + async container APIs (read_item, read_all_items, query_items) with bypass_integrated_cache and dedicated_gateway_shard_key support (including shard key validation).
Adds/updates header-focused unit tests for the new headers (sync).

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
sdk/cosmos/azure-cosmos/tests/test_headers.py	Adds tests validating dedicated gateway bypass-cache + shard-key headers, and modernizes one ValueError assertion.
sdk/cosmos/azure-cosmos/azure/cosmos/http_constants.py	Adds constants for the two new dedicated gateway headers.
sdk/cosmos/azure-cosmos/azure/cosmos/container.py	Adds new request options for bypass-cache + shard-key to sync read/query flows.
sdk/cosmos/azure-cosmos/azure/cosmos/aio/_container.py	Adds new request options for bypass-cache + shard-key to async read/query flows.
sdk/cosmos/azure-cosmos/azure/cosmos/_base.py	Emits the new headers in `GetHeaders` and adds shard key validation helper.

Copilot · 2026-02-11T17:51:35Z

        session_token: Optional[str] = None,
        initial_headers: Optional[dict[str, str]] = None,
        max_integrated_cache_staleness_in_ms: Optional[int] = None,
+        bypass_integrated_cache: Optional[bool] = None,
+        dedicated_gateway_shard_key: Optional[str] = None,


aio.ContainerProxy.read_all_items adds new parameters bypass_integrated_cache and dedicated_gateway_shard_key, but the method docstring below the signature does not document these keywords. Please update the docstring so async users discover the new options via help()/generated docs.

@copilot apply changes based on this feedback

Added missing docstrings for bypass_integrated_cache and dedicated_gateway_shard_key parameters in async read_all_items method. Also addressed other review feedback:

Fixed maxIntegratedCacheStaleness null check to support value 0

Added type validation for shard key

Updated runtime query_items docstring

Added comprehensive tests

Commit: cf89cc9

Copilot · 2026-02-11T17:51:36Z

+        if utils.valid_key_value_exist(kwargs, "bypass_integrated_cache"):
+            bypass_integrated_cache = kwargs.pop("bypass_integrated_cache")
+            feed_options["bypassIntegratedCache"] = bypass_integrated_cache
+        if utils.valid_key_value_exist(kwargs, "dedicated_gateway_shard_key"):
+            dedicated_gateway_shard_key = kwargs.pop("dedicated_gateway_shard_key")
+            validate_shard_key_value(dedicated_gateway_shard_key)
+            feed_options["dedicatedGatewayShardKey"] = dedicated_gateway_shard_key


Async dedicated gateway headers are now supported (bypass_integrated_cache / dedicated_gateway_shard_key), but there are no tests covering that the async query_items path propagates these options into request headers. Adding coverage in the async headers tests would help ensure parity with the sync behavior.

@copilot apply changes based on this feedback

Added async query_items test in test_headers_async.py that verifies bypass_integrated_cache and dedicated_gateway_shard_key headers are correctly propagated. The test uses async iteration and follows the same pattern as sync tests. Commit: cf89cc9

Copilot · 2026-02-11T17:51:36Z

    if options.get("maxIntegratedCacheStaleness"):
        headers[http_constants.HttpHeaders.DedicatedGatewayCacheStaleness] = options["maxIntegratedCacheStaleness"]

+    if "bypassIntegratedCache" in options:
+        headers[http_constants.HttpHeaders.DedicatedGatewayBypassCache] = "true" if options["bypassIntegratedCache"] else "false"


GetHeaders only sets x-ms-dedicatedgateway-max-age when maxIntegratedCacheStaleness is truthy, so a valid value of 0 (allowed by validate_cache_staleness_value) will silently omit the header. Consider checking for key presence / is not None instead so 0 is transmitted correctly.

@copilot apply changes based on this feedback

Fixed to use key presence check (if "maxIntegratedCacheStaleness" in options) instead of truthy check. This ensures value 0 is correctly transmitted as a header. Commit: cf89cc9

Copilot · 2026-02-11T17:51:37Z

        :keyword int max_integrated_cache_staleness_in_ms: The max cache staleness for the integrated cache in
            milliseconds. For accounts configured to use the integrated cache, using Session or Eventual consistency,
            responses are guaranteed to be no staler than this value.
+        :keyword bool bypass_integrated_cache: If set to True, the read will be served by the backend and won't be
+            cached in the dedicated gateway. If set to False or not provided, the integrated cache will be used if
+            configured. This option only applies to accounts configured with dedicated gateway.
+        :keyword str dedicated_gateway_shard_key: The shard key to use for the dedicated gateway request. Specifying
+            the shard key will help route the request to an instance that has cached data for this shard or bypass
+            the SQLx cache if the correlated instance is down. If not specified, the request will fall back to
+            randomly selecting an instance. This option only applies to accounts configured with dedicated gateway.


The bypass_integrated_cache / dedicated_gateway_shard_key parameter docs were added to the @overload docstring, but the runtime implementation of query_items is the def query_items(self, *args, **kwargs) definition later in the file (overload stubs are overwritten). To ensure help()/Sphinx show these new options, the implementation docstring should also document them (or the docs should live only on the implementation).

Suggested change

:keyword int max_integrated_cache_staleness_in_ms: The max cache staleness for the integrated cache in

milliseconds. For accounts configured to use the integrated cache, using Session or Eventual consistency,

responses are guaranteed to be no staler than this value.

:keyword bool bypass_integrated_cache: If set to True, the read will be served by the backend and won't be

cached in the dedicated gateway. If set to False or not provided, the integrated cache will be used if

configured. This option only applies to accounts configured with dedicated gateway.

:keyword str dedicated_gateway_shard_key: The shard key to use for the dedicated gateway request. Specifying

the shard key will help route the request to an instance that has cached data for this shard or bypass

the SQLx cache if the correlated instance is down. If not specified, the request will fall back to

randomly selecting an instance. This option only applies to accounts configured with dedicated gateway.

Copilot · 2026-02-11T17:51:37Z

+        if bypass_integrated_cache is not None:
+            feed_options["bypassIntegratedCache"] = bypass_integrated_cache
+        if dedicated_gateway_shard_key is not None:
+            validate_shard_key_value(dedicated_gateway_shard_key)
+            feed_options["dedicatedGatewayShardKey"] = dedicated_gateway_shard_key


read_all_items now accepts bypass_integrated_cache / dedicated_gateway_shard_key, but there is no test asserting these options are translated into the expected dedicated gateway headers for the ReadItems path. Adding a unit test similar to the existing read_item header tests would help prevent regressions.

@copilot apply changes based on this feedback

Added test_read_all_items_with_bypass_and_shard_key() in test_headers.py that verifies both headers are correctly set for the ReadItems path. The test follows the same mocking pattern as existing read_item tests. Commit: cf89cc9

github-actions · 2026-02-11T18:00:09Z

API Change Check

APIView identified API level changes in this PR and created the following API reviews

azure-cosmos

- Add missing docstrings for bypass_integrated_cache and dedicated_gateway_shard_key in async read_all_items - Fix maxIntegratedCacheStaleness to use key presence check instead of truthy check (supports value 0) - Add type validation to validate_shard_key_value to handle non-string types gracefully - Add bypass_integrated_cache and dedicated_gateway_shard_key docs to query_items implementation docstring - Add test for read_all_items with bypass and shard key headers - Add async test for query_items with bypass and shard key headers Agent-Logs-Url: https://github.com/Azure/azure-sdk-for-python/sessions/3867bf07-f0b6-427f-bc2d-4f2529ba716d Co-authored-by: simorenoh <30335873+simorenoh@users.noreply.github.com>

Copilot · 2026-04-13T15:53:03Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

scanning-api.github.com
- Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Initial plan

4c3e9d1

Copilot AI assigned Copilot and simorenoh Jan 26, 2026

Copilot started work on behalf of simorenoh January 26, 2026 17:23 View session

Copilot AI and others added 6 commits January 26, 2026 17:32

Final style and consistency fixes

7d04430

- Fixed string quote consistency (use double quotes throughout) - Fixed parameter indentation alignment in tests - All code now follows consistent formatting standards Co-authored-by: simorenoh <30335873+simorenoh@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add support for dedicated gateway SQLx features in Python SDK~~ Add Cosmos DB dedicated gateway bypass cache and shard key support Jan 26, 2026

Copilot AI requested a review from simorenoh January 26, 2026 17:43

Copilot finished work on behalf of simorenoh January 26, 2026 17:43

Merge branch 'main' into copilot/add-dedicated-gateway-features

59941d3

simorenoh marked this pull request as ready for review February 11, 2026 17:41

simorenoh requested a review from a team as a code owner February 11, 2026 17:41

Copilot AI review requested due to automatic review settings February 11, 2026 17:41

Copilot started reviewing on behalf of simorenoh February 11, 2026 17:42 View session

Copilot AI reviewed Feb 11, 2026

View reviewed changes

Copilot started work on behalf of simorenoh April 13, 2026 15:45 View session

Copilot finished work on behalf of simorenoh April 13, 2026 15:53

Copilot started work on behalf of simorenoh April 13, 2026 16:01 View session

Copilot finished work on behalf of simorenoh April 13, 2026 16:05

Conversation

Copilot AI commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

References

I tried to connect to the following addresses, but was blocked by firewall rules:

Comments on the Issue (you are @copilot in this section)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

simorenoh Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

simorenoh Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

simorenoh Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

simorenoh Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Feb 11, 2026

API Change Check

Uh oh!

Copilot AI commented Apr 13, 2026

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Jan 26, 2026 •

edited

Loading